Repository navigation
Open any Zarr store, not only FLUXCOM NEE - #170
Open
aivanchenk wants to merge 32 commits into
Open
aivanchenk wants to merge 32 commits into
aivanchenk wants to merge 32 commits into
Conversation
A Data panel takes a Zarr v2 or v3 URL, with presets, a variable picker and pickers for extra dimensions such as ensemble member or pressure level. The open store is mirrored in the address bar so a link reopens it. The reader finds the time, latitude and longitude dimensions in any order, decodes CF time units and calendars, applies scale factor, offset and fill values, and handles 0 to 360 longitudes and regional grids. Every store is resampled onto days by time-of-day slots so the plots and exports keep working. Stores chunked by map are capped per load with a partial-load option. Work in progress: round 1 testing found calendar, axis and copy bugs that later commits fix.
Values printed with two fixed decimals, so rainfall in kg m-2 s-1 read as 0.00 everywhere. They now keep about three significant digits, with exponents for very small or large values, while the NEE range keeps its two decimals. The line and strip axes always reached down to zero, which flattened pressure or temperature in kelvin into a line along the top. They now fit the data, and keep zero only when the data crosses it or sits close to it. Tiny negative noise in one-sided data such as rainfall no longer spreads over half the colour ramp; it is clamped to the neutral centre.
An 8-daily or weekly value filled days from its timestamp forward, ran on
into the next year and past the end of the record, and added an empty year
to the picker. Such steps are now taken as centred on their timestamp, stop
halfway to their neighbours, and never cross into another year.
A 360_day year was mapped date by date, so February 29th and 30th rolled
into March and overwrote it, and seven 31sts a year stayed empty. It is now
stretched over the real year by day of year: no value is lost and the spare
days are spread out.
Captions counted filled days as steps ("365 monthly steps"). They now count
the values the store holds for the selected years, with singular forms.
A time axis split into a thousand tiny chunks, as in MUR SST, started every request at once; the browser ran out of connections and the store was reported as having no time coordinate. Coordinates now load eight chunks at a time, and a real read failure is reported instead of being taken for a missing coordinate. Packed values outside valid_min, valid_max or valid_range are now missing, as CF defines, instead of plotting as spikes.
One-sided data keeps a tiny placeholder extent on its empty half so the scale never divides by zero. Legends printed it as -1e-10; they now read 0.
With one value a day or fewer there is no daily cycle, so the fingerprint was a single row of stripes. Such stores now get the usual year by season heatmap: one row per selected year, one column per day, week or month of the year, with the strips showing the mean of each year and the mean of each position in the year across years. The yearly scale and the exported daily-cycle table keep the old layout.
Line resolutions and fingerprint scales finer than the store's own step are
no longer offered, the finest option carries the step's name ("6-hourly"),
and a monthly store opens on its monthly view. The Local/UTC switch shows
only for hourly or finer data: shifting a 6-hourly store by whole slots
labelled Tokyo UTC+12.
Summary cards and the per-year table count and date months for monthly
stores ("Lowest month 2014-01") instead of the days a month fills. Export
copy and the CSV header name the real step, the cell caption keeps
title-case names and does not repeat the step, and the kept-in-memory box
size is capped at the store's chunk size.
A store that never answered locked the Data panel until reload. Opening now gives up after 30 seconds, and Open and the examples stay usable while a store is loading, so another one can replace it. The address field follows the store last asked for, so a failed or slow ?store= link shows its URL instead of an empty field. A mistyped variable keeps the variable field instead of hiding it. Errors are styled as errors, say which store is still showing, and read plainly for refused access (401, 403, 409), answers that are not Zarr metadata, and grids that are not regular latitude and longitude. A click outside a regional store no longer snaps to its edge cell: the title keeps the clicked point and no cell is drawn. The outermost rows of a global grid now reach the poles, so polar clicks on a Gaussian grid load.
Date.UTC reads years 0 to 99 as 1900 to 1999, so a model run starting in year 1 opened as a single year 1901 with no data. Every place that builds a date from a year now goes through one helper that does not. The year picker shows full years unless a full grid shares one century, so a single year no longer reads "20", and a long record scrolls to the selected year. Stores without a title attribute are named from their CMIP facets, or from the address without its version tag.
Counting downloads alone let a store with whole-globe chunks offer a partial load of tens of gigabytes within 400 requests (ERA5 temperature on pressure levels: about 34 GB). A load now also stops at 2 GB of decoded data, the refusal states how much the full request would take, and the partial-load button says how much it will download.
The corner map floated over the plot's right edge and the fourth summary card as the data view scrolled. Large screens now reserve its width as a right gutter whenever the page margin is narrower than the map. Also count FLUXCOM-X steps per hour again: its hour of day is its own axis, so each time sample holds 24 values.
Fingerprint labels ran into each other in the compact panel ("Jan '2Mar
'21") and on long records; a label that would touch the previous one is now
skipped. Records spanning many years label only round years (every 2, 5,
10, 25, ... years) on both plots instead of irregular ones.
A store that cannot list its arrays keeps its variable as a text field
after opening, so it can be switched or corrected. Stores are titled from a
name or dataset_id attribute when they carry no title.
Tooltips compared two points as a percentage, which means nothing for kelvin, degrees Celsius or a signed flux and blows up near zero. They now give the difference in the variable's own units. A daily point with a single value no longer reports "Std 0 across 1 step". The exported mean daily cycle keyed its rows by slot index, so a 3-hourly peak at 09:00 read as hour 3. Rows are now keyed by the hour each slot starts, and stores with one value a day leave the table out.
The focus zoom suited 0.05° cells; a 2.8° model cell then filled far more than the screen and no outline was visible. A coarse cell is now framed at the zoom where it spans about 120 px.
…t year A daily record with most days missing has a typical step of two days, so each value was spread over two days and the series gained days it never had. Steps shorter than five days are now read as daily data with gaps; only weekly and coarser steps are treated as windows. A store opened on the app's starting year (2021) even when its record ran to 2026. It now opens on its own last year, while years the reader picked still carry over when switching stores.
A regional store now opens with the map framed on its area, and the Data panel lists its coverage; units read "not given in the store" when absent. Stores often allocate their time axis years ahead (ERA5 to 2050, MERRA-2 to 2029), so the last-year default landed on an empty year. It now stops at the present year, and an untouched default that comes back empty for the chosen variable steps back up to two years. The address field kept snapping back to the previous store after a failed open; it now shows each new request's address and only the resolved root of a store that actually opened.
…L field Exports and the Data panel named the default store "NEE" at 0.049999999999999996°, where they read FLUXCOM-X-BASE at 0.05° before the refactor. The resolution is now rounded, and the default store carries its product name and "net ecosystem exchange" again. Exported cell coordinates drop float noise too. A time coordinate split into more than 64 chunks is read from its first chunk and its last value when those show an even spacing, so MUR SST opens in seconds instead of timing out. A variable typed for one store is no longer sent with a different address, typing in the address field is not overwritten by a store that finishes opening late, and a failed first open says so in the chart panel instead of describing FLUXCOM. Coverage for stores spanning all longitudes reads "all longitudes", and such stores are not framed on open.
The line joined the values on either side of a gap, so missing days or a masked period looked like a straight change. Each run of empty bins (or empty slots at hourly resolution) now carries one empty point, which the line does not draw across.
Stores like ERA5 (273 arrays) or MERRA-2 (about 470) listed every variable in a plain select with no way to search, and long group paths were cut off at the end. The variable is now a search field that matches the name, the long name or the group path, and each option shows the array name first with its groups and long name wrapping beneath.
The variable field for a store that does not list its arrays came up holding the previous store's variable. It now follows each request. Year buttons drop to two digits only for a short run within one century that does not start on a century year, so a 2000 to 2026 store no longer shows 00 to 26.
Every open logged a handful of 404s: zarrita, left to guess, tried the v2 files (.zmetadata, .zgroup, .zarray, .zattrs) on v3 stores and an array at the root of v2 ones. The store's format is now read once from its root and every node is opened in that format, and a root that consolidated metadata already lists as a group is not probed as an array.
The line caption counted empty days as daily means ("720 daily steps ·
731 daily means" for a 360-day calendar); it now counts bins with data.
Labels under the left strip of the fingerprint skip one that would touch
its neighbour instead of running together, and the exported local hour
keeps its half hour for sub-hourly stores instead of truncating to the
hour.
Stores other than FLUXCOM-X were told an empty cell might be over ocean, which is backwards for an ocean product like MUR SST. The message now names a land or ocean mask, or years outside the record. FLUXCOM-X is recognised by its address, so its vegetated-land wording no longer depends on the variable being called NEE.
After loading the first 100 days of a store chunked by map, the caption still gave the whole year's step count (1,453 for ACE). It now counts the steps the partial load covered.
|
🚀 Vercel preview deployed Preview URL: https://earth-prints-dzbuztv2w-anastasiia-s-projects10.vercel.app |
aivanchenk
force-pushed
the
feat/any-zarr-store
branch
from
November 2, 2026 22:15
1d6e1d1 to
f83e4f7
Compare
aivanchenk
added a commit
that referenced
this pull request
Nov 2, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Closes #132. Open any Zarr v2/v3 store from the new Data panel (first toolbar button) or a link:
?store=<url>&var=<name>&sel=<dim:index>.Each store was tested by its own agent, with values checked against an independent read. Screenshots are from the current build unless marked.
FLUXCOM-X-BASE NEE: default, same as
mainACE2S HiRO v2: 6-hourly, julian, chunked by map, partial load
ARCO-ERA5: too large for one load, offers the first days (no CORS, via proxy, round 3)
ESDC 8-daily: 8-day steps, °C
CESM2 monthly: noleap, 0 to 360 longitude
HadGEM3 daily: 360_day calendar
GFDL-CM4 3-hourly
CESM2 piControl: years 1 to 1200
MUR SST: ocean only, packed int16
NOAA GFS: Zarr v3 sharded, hourly
NASA POWER monthly
gridMET: regional grid (no CORS, via proxy, round 3)
CanESM5: Gaussian grid, click near the pole
Unsupported grid (IPSL tos) and no CORS (WeatherBench 2)
Local test stores: no consolidated metadata, time gaps (round 3), integer data types